Chaos and unpredictability in evolutionary dynamics in discrete time 



O 
(N 

I— J 

m 

O 

< 



> 
00 

rn 
O 



Daniele Vilone,^ Alberto Robledo,^^^ and Angel Sanchez^' 

' Grupo Interdtsciplinar de Sistemas Complejos ( GISC), Departamento de Matemdticas, 
Universidad Carlos III de Madrid, 28911 Leganes, Spain 
^Instituto de Fisica, Universidad Nacional Autonoma de Mexico, A. P. 20-364, 01000 Mexico D. F., Mexico 
'^Instituto de Biocomputacion y Fisica de Sistemas Complejos (BIFI), 
Universidad de Zaragoza, 50018 Zaragoza, Spain 

A discrete-time version of the replicator equation for two-strategy games is studied. The stationary 
properties differ from those of continuous time for sufficiently large values of the parameters, where 
periodic and chaotic behavior replace the usual fixed-point population solutions. We observe the 
familiar period-doubling and chaotic-band-splitting attractor cascades of unimodal maps but in some 
cases more elaborate variations appear due to bimodality. Also unphysical stationary solutions can 
have unusual physical implications, such as the uncertainty of final population caused by sensitivity 
to initial conditions and fractality of attractor preimage manifolds. 
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Evolutionary dynamics deals with a generic situation 
in which individuals interact and reproduce according to 
the result of that interaction, which in turn depends on 
the composition of the population (iHll- This feedback 
loop gives rise to highly non trivial phenomena, that have 
attracted the interest of the physics community from the 
dynamical systems |4|- | 7|| . statistical mechanics and 
extended systems |lll4l4| viewpoints. A particularly im- 
portant class of problems is encompassed by the family 
generically referred to as replicator dynamics [isl - flTj . If 
Xi is the frequency of type i = 1, . . . , n, x e is the 
frequency vector describing the population as a whole, 
and the interaction is given by the n x n payoff matrix 
A, then {Ax.)i is the expected payoff for an individual of 
type i and Ax is the average payoff of the population. 
Assuming that the per capita rate of growth is given by 
the difference between the payoff for type i and the aver- 
age payoff in the population, we arrive at the replicator 
equation 



Xi[iAx)i -x^Ax]. 



(1) 



Other forms have been proposed for this equation, which, 
e.g., include a denominator given by the average global 
payoff (albeit in this case the orbits of the resulting equa- 
tion are the same) or other modifications (see, e.g., 
see also [3| and references therein). However, Eq. (jlj is 
the one most often studied, and therefore we will stick to 
this form in the following. 

Among the very many nonlinear phenomena exhibited 
by the replicator equation, the appearance of chaotic dy- 
namics has been studied in a number of papers. The 
Poincare-Bendixson theorem [l^l establishes that chaos 
can only arise in a continuous dynamical system (speci- 
fied by differential equations) if it has three or more di- 
mensions. Accordingly, Skyrms showed that chaos 
does not exist for three types (also referred to as strate- 
gies) and gave examples of chaotic dynamics on strange 
attractors with four strategies. Further examples were 



provided in [2l|, [22| . Subsequently, Sato et al. [23| found 
both hamiltonian and non-hamiltonian chaos in a three- 
type version of the rock-scissors papers game [l|, by 
introducing asymmetry (payoffs depend on being the first 
or the second to interact). 

In this letter we show that replicator dynamics does 
lead to chaos in games with only two types when the 
discretized version of the equation is considered, i.e.. 



x\[{Ax^),--{x^fAxt], 



(2) 



where Xt is the composition of the population at time t. 
We will refer to Eq. ([2]) in what follows as the replicator 
mapping and can be seen as arising from situations in 
which changes per generation are not necessarily small 
[l^ . We note that this is one of several existing discrete 
versions of the replicator equations [18| and in particular 



it has been considered in [2J, |25| . As we will see below, 
the relevance of this result goes beyond the observation 
of chaos in low-dimensional systems under the required 
circumstances: Indeed, the occurrence of chaos as well as 
of periodic trajectories gives rise to unexpected behavior 
in well-known games for which so far only rest points 
have been reported. 

We will focus on the generic two-strategy game given 
by the payoff matrix 
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(3) 



where payoffs for the row player are indicated. This 
choice encompasses several of the most important dilem- 
mas arising in both biological and social applications. 
According to the values of T and 5, we have four basic 
social dilemmas: When T > 1 and > (upper-right re- 
gion in Fig. [T]) we obtain the Snowdrift game (also known 
as Chicken or Hawk-Dove) jl^, which represents a sit- 
uation in which the best thing is to do the opposite of 
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FIG. 1: Upper panel: sign of the Lyapunov exponent (bright; 
negative, i.e. periodic solutions; dark: positive, i.e. chaotic 
solutions). Lower panel: zoom of a zone of the first region. 



what the opponent does (i.e., it is an anti-coordination 
game). The region T < 1 and S < (lower-left region) 
corresponds to the Stag Hunt game [27|, a coordination 
game in which what is best is to do as the other player 
does. Finally, the other two regions define the Prisoner's 
Dilemma (T > 1 and S < 0, lower-right) [li], where D 
is the best option no matter what the opponent chooses, 
and the Harmony game [i^ , with C being now the best 
strategy. In general, practically all the studies published 
on this two-parametric representation of social dilemmas 
deal with the square T G [0, 2], S £ [— 1, 1] (referred to as 
"basic square" in what follows) as this is deemed enough 
to understand the four main classes of games, alth oug h 
other special regions can be identified (see, e.g., [30[). 
With these payoffs, the replicator mapping given by Eq. 
^ becomes 

xt+i =xt+ xtil -Xt)[S + {l-T- S)xt], (4) 

where xt now stands for the proportion of C-strategists 
in the population (the frequency of D-strategists being 
obviously 1 — xt). 

To begin our analysis, we first consider the Lyapunov 
exponent of the mapping in Eq. (|4]) as a function of T and 
S, plotted in Figure [T] As can be seen from the figure, 
below the diagonal the Lyapunov exponent is negative 
almost everywhere, whereas above the diagonal there is 
a region with intricate boundaries where the exponent 



remains positive, and outside that region it becomes neg- 
ative. Interestingly, the basic square is contained in the 
region in which the Lyapunov exponent is negative. We 
recall that a negative Lyapunov exponent is indicative of 
periodic solutions and, as will be discussed below, peri- 
ods P > 1 do exist, although always outside the basic 
square, where only the well-known fixed-point solutions 
(P = 1) are found. Perhaps this is the reason why all 
this new phenomenology has not been noticed before, 
and it indicates that capturing the richness of evolution- 
ary game theory, when viewed in discrete time, demands 
the study of a larger payoff parameter region or, alter- 
natively, of certain specific cases. In fact, as the zoom 
in Fig. [T] shows, the boundary between the chaotic and 
non-chaotic regions is very complicated, with fractal fea- 
tures, as it is regularly the case in dynamical systems. 
Another important remark is that in relation to game 
theory we have observed two types of chaos. There is 
non-physical chaos when in the chaotic region the vari- 
able Xt becomes negative or takes values larger than 1, 
which are unphysical in so far it is the frequency of C- 
strategists and therefore must be bounded by and 1; 
this takes place in the upper-left region, the Harmony 
game. On the other hand, in the Snowdrift game, we 
have observed physical chaos in the sense that the values 
of Xt remain bounded within and 1 for all times, and 
therefore trajectories represent actually realizable evolu- 
tions of populations. It is important to realize, however, 
that even non-physical chaos can be relevant in the sense 
that while trajectories become unphysical above 1 or be- 
low 0, the existence of chaos makes unpredictable in prac- 
tice where any initial condition will end, i.e., to which of 
the two absorbing states xt = oi xt = 1 will become at- 
tached. Moreover, in case the chaotic motion takes place 
only around one of these two absorbing states, so that 
we know exactly in which of them the system will freeze, 
the time r needed to reach the final configuration is sen- 
sitively dependent on the initial conditions and therefore 
unpredictable. These considerations hold in particular 
when r is large and the system stays in the physical in- 
terval [0, 1] enough time before hitting a boundary: we 
have verified that when the initial condition is close to 
an unstable equilibrium (or repellor) , this is actually the 
case. 

In order to gain more insight on how the transition to 
chaos takes place, we have focused on the case T — S — A 
and represented the Lyapunov exponent as a function of 
the new parameter A. Figure [2] shows the behavior of 
the Lyapunov exponent as a function of A. The upper 
plot clearly shows that both in the region where we found 
physical {A > 0) or unphysical {A < 0) behavior there is 
a well defined route to chaos. Furthermore, the unphysi- 
cal region reproduces the period-doubling route to chaos 
characteristic of unimodal (one hump) maps, whereas in 
the physical region we have a different but not unrelated 
route that arises because of bimodality (two extrema) of 
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FIG. 2: Upper panel: Behavior of the Lyapunov exponent 
along the diagonal T = 5", as a function of the parameter 
^ = T = S, no noise. The acronyms indicate the games found 
in the different regions: SH, Stag Hunt; HG, Harmony Game; 
BoS, Battle of Sexes [1,[23|. Lower panel: same graphics with 
noise (a = 0.005). 



the map when A > 0. For smaU to moderate A > tra- 
jectories settle into periodic and chaotic attractors gen- 
erated separately by each of the extrema of the map, but 
for sufficiently large A trajectories spread and bounce 
between both extrema and converge to new periodic and 
chaotic attractors. This is the reason why in Fig. [2] the 
Lyapunov exponent for large A > differs from the fa- 
miliar unimodal pattern shown when A < 0. The lower 
panel of Fig. [2] shows the effect on the Lyapunov expo- 
nent of additive noise in Eq. ([4]). As expected, we ob- 
serve a gradual smearing out of the fine structure due to 
the removal of periodic and chaotic-band attractors with 
increasing noise amplitude a. This amplitude fixes the 
largest period and number of bands present, 2^((t), giv- 
ing rise to the so-called bifurcation gap [sij . Importantly, 
the chaotic behavior is not restricted to the diagonal: As 
shown in Fig. [3] the boundary between the fixed point 
solution (the same as in the continuous time replicator 
equation) and the period-two orbits takes a hyperbole- 
like shape. Then, moving outwards in the T — S plane, 
boundaries between period two and period four orbits 
appear, and so on. It is important to note that the plane 
region usually considered in these studies, namely the 
basic square T € [0, 2], S € [—1, 1] (bottom left of, and 
out of. Fig. [3]) contains only fixed point solutions, i.e., 
in that region the continuous replicator equation and the 
discretized version we are studying here lead to the same 
dynamical behavior. Finally, Fig. |4] illustrates the dy- 
namics typically found in the route to chaos along the 
diagonal (T = S) of the region with physical behavior. 
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FIG. 3: Boundaries between areas of solutions with different 
periodicity. Proceeding away from the bottom-left corner, 
dots indicate curves where the maximum Lyapunov exponent 
vanishes and separate regions where the asymptotic state is a 
fixed point, a period-two orbit, a period-four orbit, etc. The 
basic square lies outside the bottom- left corner of the plot, in 
the fixed point region. 



As A is increased, we observe the convergence to the fixed 
point, continuous-like solution, or to periodic solutions 
until reaching chaos (bottom panel of Fig.|4|). This route 
to chaos appears an infinite number of times along the 
family of attractors generated by unimodal maps within 
periodic windows that interrupt sections of chaotic at- 
tractors. In the opposite parameter direction, a route 
out of chaos accompanies each period-doubling cascade 
by a chaotic band splitting cascade [l^. Bimodal maps 
with a minimum and a maximum (as in the case ^ > 0) 
display interesting variations of these attractor cascades. 

In summary, we have studied a discretized version of 
the replicator dynamics and found that there are large 
regions of the T — S parameter space of the typical so- 
cial dilemmas for which considering discrete time leads 
to behavior largely different from the continuous one. In- 
deed, beyond the restricted region which has been ana- 
lyzed before, we observe that as the parameters T and S 
increase, periodic (P > 1) and chaotic dynamics occurs. 
This is an important difference with respect to the contin- 
uous time dynamics, for which only fixed point solutions 
are found. We have also classified those novel behav- 
iors as physical and unphysical, depending on whether 
the values of the concentration of C-strategists remain 
in [0, 1] or spread beyond the boundaries of this inter- 
val. In the physical case, periodic solutions imply that 
C and D strategists vary periodically in frequency, the 
state of the system being generally mixed, containing 
both types of individuals. Such cycles have only been 
observed in the continuous replicator dynamics for rock- 
scissors-paper games, i.e., games with three strategies 
[ij-Q. The occurrence of chaotic behavior adds unpre- 
dictability to the features of the system, in complete 
contrast with the predictive character of the continuous 
case. On the other hand, periodic or chaotic behavior, 
even being unphysical, still has important consequences: 
In the time interval in which the trajectory is still con- 
tained in [0, 1], in the chaotic region it turns out that it 
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FIG. 4: Examples of time behavior for the map with T = S, 
with xt being the cooperator density vs time. From to top to 
bottom: periods one (fixed point) and two, period four, and 
chaotic solution. Values of T and S indicated. 

is not possible to predict whether, starting from a given 

initial condition, the system will end up in a population 
fully consisting of C- or D-strategists. We stress that 
our results may be relevant for real life applications, in 
many cases involving bacteria, animals or humans which 
are well described by social dilemmas that do not have 
well-determined payoffs, and often take place in discrete 
time. In principle, it is possible to test our ideas in ex- 
perimental setups, at least with human volunteers, by 
setting properly chosen payoffs and interactions as dis- 
crete events. Clearly, the appearance of periodic orbits 
and chaos is generic and will affect other strategic inter- 
actions when described by a discretized equation. Uni- 
modal maps (obtained e.g., when ^ < 0) show a rich sclf- 
affine structure of chaotic- band attractors interspersed by 
periodic windows within which multiple period-doubling 
and band-splitting cascades take place. Bimodal maps 
(obtained e.g., when A > 0) display more elaborate at- 
tractor structures. These dynamical features manifest in 
the discrete-time evolutionary dynamics addressed here 
and translate into matching patterns of population be- 
havior. 
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